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\  ABSTRACT 

The  motion  at  a  locus  on  the  projection  surface  cannot 
be  obtained  locally  from  spatio-temporal  changes  in  image 
intensity  at  that  locus.  If  the  imaged  surfaces  are  (locally) 
planar,  however,  and  generate  a  set  of  non-parallel  straight 
line  edges  in  the  image,  it  is  possible  to  obtain  estimates  of 
image  velocities  of  a  set  of  image  points  corresponding  to  the 
intersections  of  the  lines.  The  availability  of  such  a  (small) 
set  is  sufficient  to  resolve  the  underlying  optical  flow  into 
its  translational  and  rotational  components.. 
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1 .  Introduction 


If  image  motion  is  to  be  detected  by  an  (oriented)  unit 
which  is  small  compared  to  a  small  (straight  line)  contour 
segment  (along  which  the  image  brightness  is  approximately 
constant) ,  the  motion  of  this  "oriented  point"  cannot  be 
determined  by  a  purely  local  computation.  The  velocity  is 
constrained  by  a  linear  relation  but  it  cannot  be  completely 
specified  from  the  information  available  at  the  point  (Horn 
and  Schunck,  1980;  Limb  and  Murphy,  1975;  Marr  and  Ullman, 

1979).  The  linear  constraint  on  the  image  motion  at  a  locus 
(derivable  from  the  relations  between  the  spatial  and  temporal 
derivatives  of  the  image  brightness  function  at  a  point)  allows 
one  to  compute  the  direction  of  motion  if  its  magnitude  is 
known,  or  the  magnitude  of  motion  if  its  direction  is  known, 
but  both  cannot  be  obtained  from  the  information  available  at 
the  given  locus  (or  its  immediate  neighborhood)  at  au  instant. 

Marr  and  Ullman  (1979)  convolved  the  image  with  a  "Mexican 

hat"  operator  (Marr  and  Hildreth,  1979)  and  computed  the  local 

direction  of  image  motion  by  combining  the  constraints  imposed 

by  a  set  of  zero  crossing  segments  (aggregation  of  points  with 

zero  convolution  value) .  The  individual  constraints  specified 

the  sign  of  the  motion  of  the  segment,  i.e.,  the  half  plane  into 

which  the  segment  could  possibly  move.  Their  method  seems  to 

.'•**  ‘  ‘  ‘  T  J  |  '  *  I 

be  robust  but  the  degree  with  which *thlf  direction  of  {notion  (at  ■  | 

»  r.'  | 

an  image  point)  can  be  constrained  by  ^he  method  depends  directly  j 

t  >  .  :  ;  » 

.  I  y-  v 

on  the  orientation  distribution  of  the  |zero-^c?o{gsimgf  feegm^nts 
in  the  vicinity  of  the  point  in  questio^i.  ■*.  _  -  ,  ,  -  r  * 

i  - *  -  :  •  ■  1 1  t* 


u : 


%  r 


L 


'  -4 

-  5 

*■  •»  •»*  4 

-  1  |0 
O 


•  .  >4  .4 

«■*  «  r.  i’ 


j  OV  '-/• 


Horn  and  Schunck  (1980)  combined  the  (linear)  constraint 
due  to  motion  with  a  (non-linear)  constraint  imposed  by  the 
smoothness  of  the  viewed  surface,  and  iteratively  minimized 
an  error  function  expressing  both  constraints  to  obtain  a 
globally  consistent  field  of  image  velocities.  The  scheme  re¬ 
quires  that  the  visual  field  consist  of  a  dense  field  of  image 
elements  in  order  to  apply  the  constraints  derivable  from  the 
assumption  of  a  smooth  (3D)  surface.  In  fact,  both  the  schemes 
mentioned  are  closely  related,  as  they  are  based  on  the  same 
model  of  relationships  between  the  spatial  and  temporal  deri¬ 
vatives  of  the  image  brightness  function  <NOTE  1>.  Both  schemes 
are  local  in  the  sense  that  only  information  in  the  immediate 
neighborhood  of  an  image  location  enters  into  computations  at 
that  locus  (the  method  proposed  by  Horn  and  Schunck  (1980)  de¬ 
pends  on  global  convergence  properties  defined  over  the  whole 
[enclosed]  region)  <NOTE  2>.  Their  major  disadvantage  is  the 
requirement  that  the  information  be  spatially  dense  in  order 
to  be  able  to  constrain  the  direction  of  motion  (Batalli  and 
Ullman,  1980)  or  to  converge  to  a  meaningful  solution  (Horn 
and  Schunck,  1980) .  An  alternative  to  the  approach  based  on 
relations  between  the  spatial  and  temporal  derivatives  of  the 
image  intensity  is  to  solve  the  correspondence  problem  expli¬ 
citly  using  matching  (e.g.,  Barnard  and  Thompson,  1980). 


2.  The  constraint  of  (local)  planarit: 


If  the  viewed  surfaces  can  be  assumed  to  be  (locally  at 
least)  planar,  and  it  is  possible  to  extract  a  set  of  straight 
line  edge  segments  in  at  least  one  image  <NOTE  3>,  it  is  easy 
to  compute  an  estimate  of  the  image  velocities  of  a  number  of 
points  without  resorting  to  an  iterative,  relaxation-like 
process  requiring  global  consistency.  The  method  is  based 
on  the  following  two  observations. 

(a)  Under  (polar)  projection,  a  straight  line  remains 
straight  under  the  motion  transformation  on  the  image 
plane « 

(b)  The  image  velocity  component  in  the  direction  perpen¬ 
dicular  to  the  orientation  of  the  edge  at  a  point  on 
the  edge  can  be  computed  directly  locally. 

We  will  show  next  that  (b)  is  sufficient  to  determine  uniquely 
the  image  motion  of  the  intersection  of  two  (non-parallel) 
straight  lines.  Moreover,  because  of  (a),  the  motion  (in  the 
direction  perpendicular  to  the  edge  orientation)  of  an  edge 
as  a  whole  can  be  neatly  estimated  by  combining  the  motion  of 
the  individual  points  along  the  edge  (e.g.,  by  linear  least 
squares  fitting) .  The  whole  scheme  can  be  regarded  as  an  exam* 
pie  of  local/global  interaction  where  a  global  operation  (edge 
finding)  is  used  to  focus  attention  on  places  where  further 
information  gathering  (motion  detection)  can  be  performed  with 
reasonable  accuracy. 


2.1  Computing  the  component  of  edge  motion  perpendicular 

to  the  edge 

As  mentioned  above,  the  image  displacement  at  a  point 
cannot  be  computed  locally  without  introducing  additional 
constraints.  Such  local  computation  is  possible,  however, 
along  a  given  image  direction.  The  problem  is  reduced  then 
to  a  one-dimensional  problem  of  estimating  the  displacement 
of  a  known  one-dimensional  image  intensity  distribution. 

(It  is  assumed  here  that  the  distribution  does  not  change 
significantly  from  frame  to  frame,  i.e.,  that  the  change  in 
surface  orientation  due  to  the  surface  motion  has  only  a  negli¬ 
gible  effect  on  the  image  brightness  due  to  the  small  inter¬ 
frame  distance) .  The  displacement  (d)  of  an  edge  at  an  image 
point  can  easily  be  found  using,  e.g.,  the  Taylor  expansion 
about  the  given  point 


(1)  +  /(lJ+2I££AI)  }/IJU 


where  1^  is  the  first  spatial  derivative  of  the  image  intensity 
along  the  direction  i  (perpendicular  to  the  edge  orientation) , 
is  the  second  spatial  derivative  along  the  same  direction, 
and  AI  is  the  first  time  derivative  of  the  image  intensity  at  the 
given  point.  (Under  certain  circumstances  one  can  discard  all 
higher  order  terms  in  (1),  as  was  done,  e.g.,  in  Fennema 
and  Thompson  (1978)  or  Horn  and  Schunck  (1980).)  Because  a 
straight  line  remains  straight  through  its  displacement,  i.e., 
all  points  have  to  lie  on  a  straight  line  after  the  displacement, 
(it  is  assumed  that  the  image  intensity  along  the  edge  does  not 


vary  significantly  (at  least  within  the  extents  of  measured 
displacements) ) ,  the  (perpendicular)  displacements  at  all 
points  along  the  edge  are  related  by  a  linear  relation.  We 
can  thus  use,  e.g.,  linear  least  squares  fitting  or  its 
modification,  the  RANSAC  method  (Fischler  and  Bolles,  1980) 
to  obtain  an  estimate  of  edge  motion  from  a  set  of  local 
measurements.  The  planar  motion  of  a  (straight)  line  can 
be  conceptualized  as  a  translation  (its  points  all  move  with  the 
same  velocity)  and  a  rotation  (its  points  all  move  with  the 
same  angular  velocity) .  This  last  condition  constitutes  the 
linear  relation  mentioned  above;  the  displacement  at  a  point 
on  the  edge  is  a  function  of  the  distance  from  some  fixed,  but 
arbitrary  point  on  the  edge  (see  Figure  1). 

2.2  Determining  the  motion  of  an  intersection  of  two  straight 

lines 

When  two  straight  lines  intersect  and  their  motion  in  the 
direction  perpendicular  to  their  orientation  is  known  (the  mo¬ 
tion  of  the  line  as  a  whole  can  be  summarized  by  two  parameters: 
the  translation  of  an  arbitrary  point  on  it,  the  "origin",  and 
its  angular  speed)  the  image  motion  of  their  intersection  can 
be  determined  easily.  This  is  illustrated  in  Figure  2.  The 
displacements  of  the  intersection  X  in  the  directions  perpendi¬ 
cular  to  the  orientations  of  the  two  edges  are  known.  It  follows 
that  the  intersection  has  to  move  in  the  two  directions  simul¬ 
taneously.  Algebraically,  the  velocity  of  X  can  be  obtained 


by  solving  two  linear  equations  in  two  unknowns: 

(2)  dx.dl*dl  and  dx.5?=d2 

Here  dl  and  d2  (the  velocity  components  of  dx  in  the  directions 
of  dl  and  d2)  are  known  as  well  as  the  directions  dl  and  &2 
(perpendiculars  to  the  edges) .  dx  is  the  (2D)  velocity  vector 
specifying  the  motion  of  the  intersection  X. 


Discussion  and  conclusion 

Given  a  set  of  straight  lines,  a  set  of  estimates  of  image 
velocities  of  their  intersections  can  be  obtained  using  the 
method  described  above.  The  accuracy  of  the  velocity  estimates 
depends  not  only  on  the  quantization  errors  but  also  on  the 
lengths  of  the  lines;  the  longer  the  lines  the  more  accurate 
the  estimates  will  be.  The  intersections  do  not  have  to  lie 
in  the  "visible”  part  of  the  image  plane  to  be  useful. 

The  analysis  presented  here  offers  a  simple  but  powerful 
method  to  obtain  estimates  of  image  velocities  in  the  case  when 
the  (3D)  surfaces  are  planar  and  contain  a  set  of  straight  line 
edges  (as  is  usual  in  man-made  environments) .  A  has  been  shown 
(Prazdny,  1981) ,  a  small  set  of  image  velocities  is  sufficient 
to  obtain  the  parameters  of  (relative)  motion  <NOTE  4>  of  the 
surface  causing  the  image  velocities.  (The  method  employed  is 
a  minimalization  computation  relatively  insensitive  to  noise) . 


NOTES 


<NOTE  1>. 

This  can  be  seen  by  observing  that  the  relation  between  the 
image  motion  and  the  spatial  and  temporal  derivatives  of  the 
image  brightness  function  can  be  expressed  (under  certain  con¬ 
ditions;  see  the  text)  as 
3I/3t*v.VI 

where  v  is  the  image  velocity  vector,  and  I(x,y,t)  is  the  image 

brightness  function.  Using  the  convolution  values  S(x,y,t)= 

2  2 

V  G*I(x,y,t)  ("V  G*"  denotes  the  convolution  operation  with 
the  "Mexican  hat"  operator) ,  the  same  relationship  is  express- 
able  as 

3S(x,y,t)/3t=v.VS 

The  advantage  of  this  second  approach  is  that  it  guarantees 
smoothness  (and  thus  differentiability)  even  in  regions  of  sharp 
image  brightness  changes. 

<NOTE  2> . 

Fennema  and  Thompson  (1978)  report  a  scheme  based  on  the  equa¬ 
tions  used  also  by  Horn  and  Schunck  (1980).  Instead  of  a  relaxa¬ 
tion-like  scheme,  however,  they  used  a  Hough  space  approach  to 
minimize  the  effects  of  noise.  Their  scheme  is  restricted  to 
object  motions  not  containing  rotations  or  looming. 

<NOTE  3> . 

The  analysis  here  is  restricted  to  two  temporally  proximal  im¬ 
ages.  One  of  the  images  is  arbitrarily  chosen  to  be  the  "first" 
image  on  which  the  line  finding  process  operates. 

<NOTE  4> . 

Once  the  focus  of  expansion  specifying  the  translational  com¬ 
ponent  of  the  relative  motion  is  computed,  the  directions  of  the 
rotational  and  translational  image  velocity  components  at  each 
point  in  the  image  are  known.  All  information  about  the  relative 


depth  is  contained  in  the  magnitude  of  the  translational  com¬ 
ponent  of  the  image  velocity.  If  we  know  the  image  velocity 
component  perpendicular  to  the  orientation  of  the  "oriented 
point"  at  an  image  locus  we  can  project  it  onto  the  direction 
of  the  translational  component  (along  the  direction  specified 
by  the  rotational  component)  to  obtain  the  magnitude  of  the 
translational  component  (and  thus  information  about  relative 
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Figure  1 

Using  equation  (1) ,  the  displacement  (di)  in  the  direction 
perpendicular  to  the  edge  E  at  a  number  of  locations 
along  the  edge  (x)  can  be  computed  locally.  These  local 
measurements  are  then  combined  to  obtain  a  global  charac¬ 
terization  of  the  edge  motion  by  fitting  a  straight  line 
(dashed  line)  to  these  displacements. 
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